mu88 ngo

Product description



　　Behemoth-123B-v2f-Q4_K_M.gguf

　　```

　　But it returns the following error when I try loading Behemoth-123B-v2f-Q4_K_M.gguf

　　```

　　Traceback (most recent call last):

　　File "/workspace/text-generation-webui/modules/ui_model_menu.py", line 232, in load_model_wrapper

　　shared.model, shared.tokenizer = load_model(selected_model, loader)

　　^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

　　File "/workspace/text-generation-webui/modules/models.py", line 93, in load_model

　　output = load_func_map[loader](model_name)

　　^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

　　File "/workspace/text-generation-webui/modules/models.py", line 278, in llamacpp_loader

　　model, tokenizer = LlamaCppModel.from_pretrained(model_file)

　　^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

　　File "/workspace/text-generation-webui/modules/llamacpp_model.py", line 85, in from_pretrained

　　result.model = Llama(**params)

　　^^^^^^^^^^^^^^^

　　File "/workspace/text-generation-webui/installer_files/env/lib/python3.11/site-packages/llama_cpp_cuda/llama.py", line 372, in init

　　_LlamaModel(

　　File "/workspace/text-generation-webui/installer_files/env/lib/python3.11/site-packages/llama_cpp_cuda/_internals.py", line 55, in init

　　raise ValueError(f"Failed to load model from file: { path_model}")

　　ValueError: Failed to load model from file: models/Behemoth-123B-v2f-Q4_K_M.gguf

　　```

　　I assumed that I needed to follow the instructions [here](https://huggingface.co/TheBloke/CodeLlama-70B-Python-GGUF#q6_k-and-q8_0-files-are-split-and-require-joining) to join the files together. I am using oobabooga/text-generation-webui on runpod.","html":"

　　Hey, I tried to combine "Behemoth-123B-v2f-Q4_K_M-00001-of-00002.gguf" and "Behemoth-123B-v2f-Q4_K_M-00002-of-00002.gguf" by using the following commands:

　　But it returns the following error when I try loading Behemoth-123B-v2f-Q4_K_M.gguf

　　I assumed that I needed to follow the instructions here to join the files together. I am using oobabooga/text-generation-webui on runpod.

　　","updatedAt":"2024-11-28T16:43:18.456Z","author":{ "_id":"65165701c2aedaa7e6a544ed","avatarUrl":"/avatars/ff0668156065ceb0488c4449faa91e8d.svg","fullname":"Frank Smith","name":"Frank2478","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":1,"identifiedLanguage":{ "language":"en","probability":0.41789233684539795},"editors":["Frank2478"],"editorAvatarUrls":["/avatars/ff0668156065ceb0488c4449faa91e8d.svg"],"reactions":[],"isReport":false}},{ "id":"67489f8baaf6c42d3aa8c0c5","author":{ "_id":"633f2dbe33ba83e00bd85389","avatarUrl":"/avatars/6418d485f21b382af23459ba11076d76.svg","fullname":"Daa a","name":"linkpharm","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2024-11-28T16:51:23.000Z","type":"comment","data":{ "edited":false,"hidden":false,"latest":{ "raw":"Point koboldcpp at the first part with part 2 in the same dir. ","html":"

　　Point koboldcpp at the first part with part 2 in the same dir.

　　","updatedAt":"2024-11-28T16:51:23.319Z","author":{ "_id":"633f2dbe33ba83e00bd85389","avatarUrl":"/avatars/6418d485f21b382af23459ba11076d76.svg","fullname":"Daa a","name":"linkpharm","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"identifiedLanguage":{ "language":"en","probability":0.8904920816421509},"editors":["linkpharm"],"editorAvatarUrls":["/avatars/6418d485f21b382af23459ba11076d76.svg"],"reactions":[],"isReport":false}},{ "id":"6748c45824a9921776ac0a81","author":{ "_id":"61c47e9c71a107e9d80e33e3","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1640356718818-61c47e9c71a107e9d80e33e3.jpeg","fullname":"Henky!!","name":"Henk717","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":85,"isOwner":false,"isOrgMember":false},"createdAt":"2024-11-28T19:28:24.000Z","type":"comment","data":},"numEdits":0,"identifiedLanguage":{ "language":"en","probability":0.9332345724105835},"editors":["Henk717"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1640356718818-61c47e9c71a107e9d80e33e3.jpeg"],"reactions":[],"isReport":false}},{ "id":"6749247658e8d2842476714d","author":{ "_id":"65165701c2aedaa7e6a544ed","avatarUrl":"/avatars/ff0668156065ceb0488c4449faa91e8d.svg","fullname":"Frank Smith","name":"Frank2478","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2024-11-29T02:18:30.000Z","type":"comment","data":{ "edited":false,"hidden":false,"latest":{ "raw":"Thanks, but is there a solution for text-generation-webui? ","html":"

　　Thanks, but is there a solution for text-generation-webui?

　　","updatedAt":"2024-11-29T02:18:30.317Z","author":{ "_id":"65165701c2aedaa7e6a544ed","avatarUrl":"/avatars/ff0668156065ceb0488c4449faa91e8d.svg","fullname":"Frank Smith","name":"Frank2478","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"identifiedLanguage":{ "language":"en","probability":0.8101018667221069},"editors":["Frank2478"],"editorAvatarUrls":["/avatars/ff0668156065ceb0488c4449faa91e8d.svg"],"reactions":[],"isReport":false}}],"pinned":false,"locked":false,"collection":"discussions","isPullRequest":false,"isReport":false},"repo":{ "name":"TheDrummer/Behemoth-123B-v2.2-GGUF","type":"model"},"activeTab":"discussion","discussionRole":0,"watched":false,"muted":false}">